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Abstract 

In the process of eye tracking , a subject may focus on a point for a longer time, we call it fixation points, the process between 
fixation points is a saccade. Here, we investigate into the correlation between eye-tracking data and 
EEG(Electroencephalogram) rhythms and aim to explore the EEG rhythm feature related to eye fixation. Furtherly, we try to 
design a new brain-computer interface system to predict the regions of interest when viewing nature images using EEG 
rhythms. This BCI system can be used on mobile devices for fixation detection. Our experiments show alpha-band EEG rhythm 
is related to fixations and EEG rhythm which makes slight contribution to ROI prediction. 
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Introduction 

When exploring a scene, we systematically move our eyes to produce a discrete sequence of fixations, gathering 
visual information in each instance of the sequence. Two noninvasive methodologies — event-related potentials 
(ERPs) and eye tracking — have largely contributed to our understanding of visual processing inside human visual 
systemfl]. 

There are two kinds of methods in studying viewing perception, task-rele vented and task free. In a task relevented 
study we give subjects different tasks, and the fixations of a subject are differently distributed in tasks. In a task 
free study, subjects are allowed to view a object freely and they view what they are interested in. In our study, we 
focus on an atom activity of a subject-whether a subject is interested in a part of the picture or not, and we 
attempted a way to decode the interested regions(ROI) in a picture. 

The study is mainly focused on cognitive activity of subjects. In the process of eye tracking , there are fixations and 
saccades. The majory of fixation related brain activity study is based on ERPs. Target detection elicits a consistent 
response appearing around 300 ms after stimulus onset[2]. Picture detection and text detection use eye fixation- 
related potential (EFRP) . EFRP is a type of ERP measuring electrical brain activity in response to eye fixations, but 
contrary to ERPs, the subjects are allowed to move their eyes during the task performance. The EFRPs are averaged 
at the offset of saccades, not at the onset of stimulus events, as in the ERP studies [3]. EFRP needs an average of 
fixation points, this makes it hard to synchronously study perception feature with eye tracking . EEG rhythms is 
another way to detect congnitive activity while viewing. EEG rhythm is easily combined with synchronous eye 
tracking signal. Previous EEG studies have shown that attention-related changes in anticipation of forthcoming 
visual events are reflected in sustained modulations of oscillatory alpha-band (8-14 Hz) activity[4]. We extract 
attention related to alpha-band rhythm as feature. 

HMMs have been successfully applied in speech recognition, anomaly detection in video surveillance, and 
handwriting recognition. In the studies related to eye movement analysis, Salvucci and Goldberg used HMMs to 
break down eye trajectories into fixations and saccades [5]. HMM analyzed time series with hiden parameters 
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which matches our study. Fixations are not statistic independent, they are related to those previous fixation points. 
We use HMM to decode ROIs with alpha-band rhythm. 

Methods 

Apparatus 

Stimuli were presented in a CRT monitor, at a screen resolution of 1,024*768 pixels and a refresh rate of 75Hz. 
Participants sat in a comfortable chair at 60 cm from the screen. Eye Tracking signal is collected with Tobii X120 
with sample frequency of 120Hz. EEG signal is collected with G-TEC of 256Hz sample frequency. 

Target Finding Experiment: Visual Search Task 

This experiment uses seven categories of objects, including animal, architect, bag, botany, car, computer, and gun. 
Each of the object is a 4:3 portion picture attached to grey backgroud. 25 chosen objects joint into one 1024 * 
768(pixels) picture which is 5*5 distributed. In each picture, 5 out of the 25 objects are targets. 5 objects are in the 
same category, the rest of 20 objects are equally chosen from other 6 catagories. 

One subject takes 5 sessions in a target finding experiment. Each session has 21 trials. So, one subject takes 105 
trials finally. In one trial, we get four parts. Part I, prompt objects to find(3 seconds); Part II, fixation point(l 
second); Part III present main picture(5 seconds); Part IV, grey screen to separate this trial from next trial(l second). 
We got 10 seconds in a trial. 

Eye Tracking Data Processing 

Missing data are always available in raw eye tracking data, we apply linear interpolation to figure out the position 
of the missing points. We use a moving average filter with a window of 5 points to smooth the eye tracking data 
and remove noise points. 

In order to separate saccades from fixation points, we use the Velocity-Threshold Identification (I-VT) classifier. 
The general idea behind an I-VT filter is to classify eye movements based on the velocity of the directional shifts of 
the eye. If it is above a certain threshold, the sample for which the velocity is calculated is classified as a saccade 
sample and below it is seen as part of a fixation[5]. 

There are often plural fixations points in one picture area in adjacent time. These points should belong to one 
fixation which simplifies explanation of fixation points. In our study, we merge fixations that have a time interval 
below 75ms to form a larger stronger fixation point which is finally taken as fixation feature. At the same time we 
discard fixation points with duration less than 30ms to get strongest features. 

EEG Data Processing 

We get 64 channels EEG data via G-TEC. Data processing is applied with matlab 2010b. A 8 th order butterworth 
filter with a range of 0.1Hz~100Hz is used to preprocess EEG data. Wavelet transform is used to select 8~14Hz 
time-frequency signal synchronous with eye tracking features. 

Synchronous Processing: Eye Tracking Data and Alpha-band Rhythm 



FIGURE 1 TWO STATE TRANSFER MODEL 

We first discretizate the energy of alpha-band rhythm to three stages: Low, Mid, High. We take portion as 
discretization standard. After sorting Fixations by EEG Rhythm, 33% fixations with highest energy belong to stage 
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"High"; 33% fixations with lowest energy belong to stage "Low"; The rest of 33% fixations belong to stage "Mid". 
Discretizated Rhythm Energy is used to describe observable states. 

Hidden States are described by this fixation point is in region of interest or not. So a two-state transfer model is 
created(FIGURE 1). 

Results 

Target Finding Experiment 

In this period. Occipital EEG Rhythm is used for alpha-band rhythm extraction. There are 105 trials for one subject. 
We implement a 5-fold cross validation. Four out of five are training sets, and one is testing set. This 5-fold cross 
validation is looped for 100 times in order to get stable results. Each fixation point is taged if it is a target fixation or 
a non-target fixation. HMM predits the ROI probability of each fixation. We take prior probability as threshold, 
after decoding with HMM, if the probability of a fixation is above threshold, it is classified to target(ROI), 
otherwise it is anon- tar get(NON-ROI) point. As FIURE 2 showes, blue bar is accuracy of subject. Red bar stands for 
the prior probability of a subject's target points. 



FIGURE 2 ACCURACY OF 5-FOLD CROSS VALIDATION 

The accuracy of HMM decoding is a bit higher than pior probability. TABLE 1 shows the detailed accuracy of each 
subject. 


TABLE 1 ACCURACY SUB JECTl (LEFT) AND SUB JECT2 (RIGHT) 



Non-target 

Target 

Non-target 

51.38% 

48.62% 

Target 

48.61% 

51.39% 



Non-target 

Target 

Non-target 

54.24% 

45.76% 

Target 

45.76% 

54.24% 


Hmm Decoding 

TABLE 2 and FIGURE 3 give transform matrix and confusion matrix of HMM. Energy of EEG Rhythm is in 
observable state. ROI and NON-ROI are in hiden states. In transform matrix, NON-ROI to NON-ROI and ROI to 
ROI probability is higher than cross state probability. In confusion matrix, low enegy alpha rhythm fixations are 
more likely to be a ROI points and high energy fixations are more likely to be NON-ROI points. 

TABLE 2 THE TRANSFORM MATRIX OF NON-ROI AND ROI POINTS 


Subject 


NON-ROI 

ROI 

Subjectl 

NON-ROI 

56.47% 

43.53% 

ROI 

40.07% 

59.93% 

Subject2 

NON-ROI 

67.13% 

32.87% 

ROI 

47.36% 

52.64% 
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Conclusions 

In our study we use EEG Rhythms instead of EFRPs to study fixation points of region of interest. EEG Rhythms is 
better at presenting sequence signal, while EFRPs have a better accuracy of analysing one fixation point. There are 
differences between rhythms of ROIs and NON-ROIs, ROIs get lower average energy than NON-ROIs. We can see 
the trend of EEG rhythms but decode accuracy is not very good. A better model of decoding needs to be explored. 
This EEG Rhythm based on BCI system will be very useful on mobile devices for fixation detection 
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